1. The Field of the Invention
The present invention relates generally to remote sensor technology, and more particularly, but not necessarily entirely, to an airborne remote sensor array capable of producing georectified three dimensional (3D) digital imagery in real time, using a lidar based system.
2. Description of Related Art
The prior art is characterized by photogrammetric systems which use stereographic image-matching techniques. Such systems produce 2.5-D imagery by draping aerial imagery onto previously obtained topographical data. The topographical data is acquired by photogrammetric measurements obtained through stereoscopic viewing of two images. Computerized image matching techniques can automatically measure the range to objects within an image and thereby create a topographic model on which the image can be coregistered and draped. These prior art methods require computational intensive post-processing and cannot be performed in real time.
The prior art also includes time-of-flight topographic lidar sensor systems, which have been developed and are used in conjunction with passive imaging sensors deployed as separate instruments on aircraft. The goal of such systems is to produce orthoimages by combining the lidar range data and the passive images such that each image pixel corresponds with known geographic coordinates.
However, the orthorectification accomplished by the prior art systems requires extensive computational post-processing because the data sets produced by the different sensors are not time synchronized. Rather, the prior art is characterized by systems in which the lidar system collects data at one time and from one direction, while the imaging sensors collect data at a slightly different time and from a slightly different direction. Following the removal of parallax and co-registration of the data sets, the imagery is then associated with the lidar data to produce an interpolated three-dimensional (hereinafter 3D) aerial image. After the 3D aerial image is produced, the image can be incorporated into a geographic information system for viewing from various perspectives including an orthoprojection. However, this process requires the surface application of complex analytic equations which in combination require a large amount of computation that cannot be completed in real time without the assistance of substantial parallel processing. The resulting time delay in acquiring such 3D aerial images is prohibitive for time-critical applications encountered in both civilian and military settings.
The publication identified as Clapis et al. xe2x80x9cReal-time image processing and data fusion of a two-channel imaging laser radar sensor,xe2x80x9d SPIE Vol. 1633 Laser Radar VII (1992) pp. 281-287 (hereinafter xe2x80x9cClapis et al. publicationxe2x80x9d), discloses a real-time image processing and data fusion of a two-channel imaging laser radar (lidar) sensor. The first channel is the lidar return intensity image or lidar image. The second channel is a lidar range image. These two data sets are derived from the same laser shot, the intensity image from the laser return signal intensity, and the range image from the time-of-arrival of the laser return signal. The two data sets are inherently synchronized and are therefore already fused at the pixel level, and are not derived as a combination of a passive imaging sensor that is separate from the lidar sensor. However, the Clapis et al. publication does not suggest the concept of combining passive spectral images in the final image. Applicant""s invention is the first to synchronize a passive imaging sensor with a lidar sensor to enable real-time pixel-level fusion of the two data sets.
Canadian Patent No. 2,105,501 by Jasiobedzki et al. (hereafter xe2x80x9cxe2x80x2501 Canadian patentxe2x80x9d) discloses an invention which coaligns a lidar sensor along the same optical axis as an imaging sensor to allow a robot to sight the lidar sensor in a particular direction, and allow the robot to determine what part of the environment the beam is currently striking. This invention successfully removes parallax between the two sensors when they are static relative to the target. However, the xe2x80x2501 Canadian patent fails to synchronize the two sensors and therefore fails to remove parallax introduced by moving objects or a moving sensor.
The prior art is characterized by the above-described and other disadvantages that are addressed by the present invention. The present invention minimizes, and in some aspects eliminates, the above-mentioned failures, and other problems, by utilizing the methods and structural features described herein.